1. Field of the Invention
This invention relates generally to software development and more specifically to testing software components.
2. Description of Related Art
It is often desirable to test software as it is being developed, modified or optimized. Software may be tested as a “black box,” meaning that inputs are applied to stimulate the software and the outputs produced in response are observed and compared to the expected response for properly functioning software. Testing in this way usually relies on inputs and outputs applied using the existing external interfaces to the software—i.e. the interfaces that would be used to exchange data with the software as it operates.
FIG. 1A illustrates such a testing approach. A development environment 110 and a host environment 120 are used. Tests are prepared in the development environment 110. The program under test operates in host environment 120. Host environment 120 includes interfaces 122 and 124.
The development environment is often a process established on the computer of a developer writing or testing programs. The development environment may have multiple components, such as one component that provides tools to help a developer write programs or tests and another component that provides a test environment for executing tests. The host environment is often a process on the same or a different computer. The host environment is created by the operating system when the host application is started for testing either by a human user or by the development environment.
Development environment 110 contains an agent 112. Agent 112 is a software program that controls the timing and sequencing of individual tests. Agent 112 may also compare results of those tests to expected results and may process or report the results of tests to a human developer or some other part of the system. Agent 112 calls tests, such as 114A, 114B and 114C. In the illustrated embodiment, agent 112 invokes a test through a test adapter 113. Each test to be executed may have its own test adapter providing agent 112 with a standard interface to all tests. Each of the tests applies inputs and observes outputs from a program under test running within a host environment 120.
Host environment 120 may be created by an application that can accept add-in programs through an extensible interface. For example, applications in the Microsoft® Office® suite of applications, such as the Word® word processing application or the Excel® spreadsheet application, accept add-in programs. Add-in programs may be user written or may come from some other source. The add-in programs may be formed from one or more units. In addition, the host environment may include one or more units that also may be tested. In FIG. 1A, the units in host environment 120 to be tested are illustrated generically by units, such as 126A, 126B and 126C.
The units may be tested using black-box testing. For black box testing, the program under test is accessed through an external interface, such as interface 122 or 124. For example, FIG. 1A shows tests 114A . . . 114C applying inputs through interface 122 and obtaining a result through interface 124. By comparing the actual result to an expected result, the performance of the program under test may be validated.
A drawback of testing software as a black box is that it may require many inputs or combinations of inputs to fully exercise all of the units. In addition, if a fault is detected because the expected outputs are not observed in response to a specific set of inputs, it may be difficult to identify the specific unit of code that needs to be modified to correct the fault.
An alternative approach to testing application programs is shown in FIG. 1B. In FIG. 1B, a unit under test, such as unit 126A, is loaded into the test environment 130. The agent 112, sometimes called a “test harness,” similarly controls the selection and sequencing of tests to be performed, such as by calling a test 154. In the example of FIG. 1B, though, the tests are performed on a single unit under test and are performed in the development environment or test environment.
Development environment 130 contains tools to aid the developer in writing tests. For example, the Microsoft® Visual Studio® development environment contains tools that allow a developer to author tests, control execution of those tests and analyze the results. One feature of the tools found in the Microsoft® Visual Studio® development environment is known as Intellisense. The Intellisense feature provides context sensitive help to the user of a code editor. For example, as a user is authoring code that is to access a variable in a program, the Intellisense feature can provide the user with a list of variables in that program, allowing the user to quickly select the correct variable to use or to have the code editor correctly complete instructions as the user begins to input them.
A test developer using the Visual Studio development environment to create tests for the test configuration shown in FIG. 1A can use the Intellisense feature if a project in the host environment containing the code under test is referenced in the project in the development environment containing the test program. However, only public interfaces from a project are available in another project that references it. Accordingly, the Intellisense feature is not available for accessing non-public portions of the units under test.
It would be desirable to have an improved method of testing units of software.